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ABSTRACT 

This study investigated whether a self-efficacy course would 
raise English-as-a-Second-Language learners' English ability by raising their 
self-efficacy. It also examined the reliability and validity of a self- 
efficacy questionnaire and English test. Participants were 293 Japanese 
college freshmen. Researchers developed two versions of an English test (in 
English) and a self-efficacy questionnaire (in Japanese). The reliability of 
the self-efficacy pre- and post-questionnaire was calculated using the 
Cronbach alpha. The English pretest-posttest reliability was calculated using 
the K-R20. Each of four instructors graded the writing and speaking portions 
of the English test twice. The speaking interviews were recorded and played 
back only once during the second grading. Results indicated that differences 
between the pre-questionnaire and post-questionnaire scores were significant. 
The results suggested that the self-efficacy questionnaire and English test 
were reliable and valid, noting that students' self-efficacy and English 
ability increased over the period of instruction. Eight appendixes include 
the course objectives, the English test, the Japanese self-efficacy 
questionnaire, the English translation of the self-efficacy questionnaire, 
pretest and posttest writing and speaking results, writing and speaking score 
guides. (Contains 13 references, 9 tables, and 2 figures.) (SM) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



3 , 73 ( 55 " 



@ 



Creating a Reliable and Valid Self-Efficacy Questionnaire and English Test to Raise Learners’ 



L2 Achievement via Raising their Self-Efficacy 



Stephen A. Templin, Timothy C. Guile, and Takanobu Okuma 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 



Meio University 




U.S. DEPARTMENT OF EDUCATION 
Office ol Educational Research and Improvement 




originating it. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



□ Minor changes have been made to 
improve reproduction quality. 



1 



Author Note 



Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position cr policy. 



This paper was presented at the Japan Association for Language Teaching 27 th Annual International 



Conference on Language Teaching. We would like to thank Seido Language Institute for their 
permission to reproduce materials within their copyright. Comments regarding this article should be 
addressed to Stephen A. Templin, Meio University, International Cultural Studies Division, 1220-1 
Biimata, Nago-shi, Okinawa, JAPAN 905-0005. E-mail: steve@ics.meio-u.ac.iD . Work fax: 
0980-52-4640. 



Stephen Templin has authored and co-authored books and numerous articles relating to TESOL 
(Teaching English to Speakers of Other Languages) and applied linguistics. Timothy Guile is a 



tradition. Takanobu Okuma lectures at Meio University and researches autonomy in second language 



Biographical Sketch 



language researcher who has authored works in general linguistics and Native American oral 



acquisition. 



1 




BEST COPY AVAILABLE 



2 



Abstract 



It is claimed that people with high self-efficacy outperform people with low self-efficacy. This study 
researches the reliability and validity of a self-efficacy questionnaire and English test. It also asks 
whether or not a self-efficacy course leads to higher self-efficacy and higher English achievement. 
Two hundred and ninety=three Japanese university students participated in this research. The 
questionnaire and English test proved reliable at p < .05. A t-test was used to determine if there was 
any significant difference before the instruction began at the beginning of the semester and after the 
instruction was completed at the end of the semester. Students' self-efficacy and English test scores 
showed a statistically significant gain at the end of the semester. 
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Self-Efficacy 



Self-efficacy is a person’s belief in how well he/she can accomplish a task or group of 
tasks (Bandura, 1997; Templin, Shiroku, & Taira, 1999). Learners who believe they can leam a 
language are more likely to leam a language than learners who believe they cannot leam a language. 

It is often asked, how does selfrefficacy differ from selfrconcept, selfrconfidence, and 
self-esteem (Coopersmith, 1967; Shavelson, Hubner, & Stanton, 1976; Yule, Yanz, & Tsuda, 1985)? 
Self-efficacy research differs from the other self-phenomena research in five areas; (1) judgement of 
capabilities, (2) multiple dimensions, (3) contexts, (4) mastery-criterion, and (5) pre-task 
measurements (Zimmerman, 1995). 

The first difference is that other self-phenomena researchers usually judge how people feel 
about themselves (i.e., personal qualities) whereas self-efficacy researchers judge how well people 
believe they can accomplish a task or group of tasks, Second, although self-phenomena researchers 
may not pay attention to task dimensions, self-efficacy researchers do. For example, self-efficacy 
researchers understand that learners may strongly believe they can read English but may not believe 
they can speak English — or a learner who believes she can speak English fluently may lack a belief 
in her ability to speak English using accurate syntax. 

Third, other self-efficacy researchers tend to ignore information about how various 
contexts affect people’s beliefs in their capabilities, but self-efficacy researchers understand that, for 
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example, although introducing oneself at a party of friends and introducing oneself in a meeting of 



strangers are similar acts, the context itself can affect how well a person believes he can make a 
self-introduction. 

Fourth, other self-phenomena researchers compare how people feel about themselves 
compared with how other people feel about themselves. In contrast, selffefficacy researchers look at 
how well people believe they can perform a task/tasks based on some criterion, usually measured by 
numerical values, Just because a person feels better about her English ability than others around her 
feel about their own ability does not automatically mean that she has a strong belief in her own 
ability to perform an English task or group of tasks. Self-efficacy researchers are interested in 
measuring her performance and comparing it to the strength of her belief in her ability. 

Fifth, other self-phenomena researchers measure the self-phenomenon before their 
subjects perform the task, after they perform the task, or the subjects may not perform the task at all 
Self-efficacy researchers measure learners’ beliefs in their capability to perform a task before they 
perform the task, and then they measure how well the person actually performed the task— this 
process is necessary in order to predict learners’ performance on a task or group of tasks. 
Predicting Performance 

Although other self-phenomena researchers have failed to predict performance, 
self-efficacy researchers have successfully predicted that people with high self-efficacy exert more 
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attention, effort, persistence, and strategies than those with lower self-efficacy, and when those with 
low self-efficacy fail, they often blame their failure on everything except their own shortcomings 
(Bandura, 1 997). Also, people with high self-efficacy set more challenging goals for themselves than 
people with low self-efficacy. Because challenging goals lead to enhanced performance (Griffee, 
1997a; Griffee & Templin, 1998; Locke & Latham, 1990), people with high seffiefficacy outperform 
people with low self-efficacy. 

Too much self-efficacy can get a person killed in dangerous activities such as rock 
climbing, but a lack of self-efficacy in less dangerous activities can lead to a lifetime of regret: 
“educational opportunities forsaken, valued careers not pursued, interpersonal relationships not 
cultivated, risks not taken, and failures to exercise a stronger hand in shaping one’s life course” 
(Bandura, 1997, p. 71). 

Self-Efficacy Course 

Psychology researchers have shown that first language (LI) learners with high 
self-efficacy in speaking (Zimmerman & Martinez-Pons, 1990), listening (Schunk & Rice, 1984), 
writing (Zimmerman & Bandura, 1994), and reading (Schunk & Rice, 1993) outperform LI learners 
with low self-efficacy. These researchers were also able to raise students’ LI performance by raising 
their self-efficacy. It seems odd that similar studies in L2 acquisition are scarce; thus, research such 
as our study is needed to fill the gap. 
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In an intensive summer English class, Templin, Shiroku, and Taira (1999) piloted a 



self-efficacy syllabus to raise second language (L2) learners’ English ability via raising their 
self-efficacy. This led to the creation of a self-efficacy course for freshman Japanese students 
studying English as a foreign language (Templin & Guile, 2000). 

We intend to evaluate our students’ achievement in relation to the goals and objectives of 
our English I course. In addition, we intend to measure students’ self-efficacy improvement. In order 
to measure our students’ self-efficacy and achievement in the course, we created a reliable and valid 
English test and self-efficacy questionnaire. 

The Study 

Research Statement 

We guided our research by asking ourselves the following questions: 

(1 ) What is the reliability and validity of the self-efficacy questionnaire? 

(2) What is the reliability and validity of the English I test? 

(3) If the self-efficacy questionnaire and English I test are reliable and valid, have our 
students’ self-efficacy and English ability significantly increased by the end of the 



English I course? 



Method 



Participants 

The 293 students who participated were mostly 1 8 year-old freshmen (about half male and 
half female) enrolled in English I, a required course. Although the enrollment for this course was 
345 students, we dropped 52 students from the study who were not present on both days the pre= and 
posttests and questionnaire were administered. The students came from three departments: 
International Cultural Studies, Business and Information Systems, and Tourism. The classes were 
one and a half hours long and convened twice a week for fifteen weeks. English proficiency scores 
were not available for the students. 

Instruments 

We created two instruments: an English I test (A and B versions, see Appendix B) and a 
self-efficacy questionnaire (see appendices C and D). For both instruments, we looked at available 
theories describing our constructs (Griffee, 1 997b) and examined similar instruments created by 
other researchers (see Templin, 1999, for a discussion of self-efficacy instruments). We felt our 
students would be able to understand English I test items written in English, but we did not feel they 
could understand the self-efficacy questionnaire items written in English, so we decided to write the 
English test in English and the self-efficacy questionnaire in easy-to-understand Japanese. We chose 
a seven-point Likert scale for the questionnaire and brainstormed which items to include in the 
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questionnaire from the English I goals and objectives. 



Next, we wrote numerous items for each instrument and held frequent meetings to make 
revisions. Then we piloted the instruments. After piloting, we analyzed the English I test using item 
analysis. We scrutinized items with low point biserials, eliminating eight items that did not 
discriminate well between students who had achieved the course objectives, and students who had 
not. We also re-worded one item to improve clarity. Then we modified the test further by giving 60 
percent of the points possible to the four skills section (which we wanted to emphasize the most) and 
40 percent to the other areas (e g., syntax). For the piloted self-efficacy questionnaire, we correlated 
the average of each item with the overall average and eliminated items with low correlations and 
re-worded one item. 

We also created a content validity judgement scale for the English I test and the 
self-efficacy questionnaire (see example in Brown, 1996, pp, 235-236), rated each item, and 
compared our ratings. At this point, we felt satisfied with the English I test (Appendix B) and the 
self-efficacy questionnaire (appendices C and D) and decided not to make any more changes. 
Reliability of the Instruments 

The reliability of the self-efficacy pre-questionnaire and post-questionnaire was calculated 
using the Cronbach alpha and is reported in tables 1 and 2. 
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Table 1 : Pearson Correlation (One-Tailed) and Cronbach Alpha of Self-Efficacy Pre-Questionnaire 
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Q2 


.61* 






















Q3 


.53* 


.46* 




















Q4 


.47* 


.40* 


.64* 


















Q5 


.48* 


.43* 


.48* 


.56* 
















Q6 


,41* 


.39* 


,38* 


,53* 


,57* 














Q7 


.44* 


.36* 


.50* 


.72* 


.55* 


.58* 












Q8 


.47* 


.36* 


.62* 


.56* 


.53* 


.47* 


.75* 










Q9 


.52* 


.52* 


.41* 


.45* 


.58* 


.49* 


.47* 


.48* 








Q10 


.42* 


.39* 


.40* 


.45 


.54* 


.53* 


.50* 


.48* 


.54* 






Qll 


.41* 


.39* 


.34* 


.38* 


.36* 


.38* 


.36* 


.35* 


.42* 


.38* 




Q12 


.46* 


.37* 


.38* 


.35* 


.34* 


.34* 


.38* 


.44* 


.43* 


.39* 


.50* 


Q13 


.39* 


.40* 


.37* 


.42* 


.41* 


.42* 


.38* 


.39* 


.41* 


.42* 


.37* 



* Correlation is significant at the .05 level (one-tailed). 
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Table 2: Pearson Correlation (One-Tailed) and Cronbach Alpha of Self-Efficacy Post-Questionnaire 
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.45* 












Q3 


.48* 


.38* 










Q4 


.46* 


.37* 


.67* 








Q5 


.42* 


.31* 


.44* 


.44* 






Q6 


.46* 


.28* 


.47* 


.43* 


.57* 




Q7 


.47* 


.30* 


.55* 


.63* 


.42* 


.42* 


Q8 


.39* 


.26* 


.65* 


.59* 


.38* 


.43* 


Q9 


.29* 


.34* 


.39* 


.33* 


.50* 


.43* 


Q10 


.40* 


.44* 


.39* 


.39 


.38* 


.37* 


QU 


.33* 


.27* 


,28* 


.28* 


,38* 


.42* 


Q12 


.33* 


.31* 


.33* 


.33* 


.44* 


.44* 


Q13 


.31* 


.24* 


.33* 


.35* 


.33* 


.43* 



Q7 Q8 Q9 Q10 Qll Q12 Q13 



.69* 








.37* 


.40* 






.44* 


.35* 


.33* 




.19* 


.26* 


,25* 


,36* 


.32* 


.30* 


.33* 


.39* .41* 


.33* 


.32* 


.32* 


.37* .48* 



* Correlation is significant at the .05 level (one-tailed). 



Cronbach alpha = .89 



All questions in both tests exhibited significantly high correlations at the .05 level (one-tailed). The 
Cronbach alpha for the pre-questionnaire was .91 and the post-questionnaire was .89. In other words, 
the questionnaire was reliable. 

The English pretest and posttest reliability was calculated using the K-R20 shown in Table 



3. 
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Table 3: Descriptive Statistics of English Pretest and Posttest (A and B Versions) 


Statistics 


Pretest A 


Posttest A 


Pretest B 


Posttest B 


N 


293.00 


293.00 


293.00 


293.00 


k 


36.00 


36.00 


36.00 


36.00 


M 


18.71 


27.55 


17.88 


27.33 


Low 


2.00 


14.00 


3.00 


17.00 


High 


34.00 


36.00 


30.00 


35.00 


SD 


5.35 


4.53 


4.91 


4.31 


K-R20 


.77 


.75 


.74 


.74 



Pretest version A, Posttest version A, Pretest version B, and Posttest version B showed .77, .75, 

.74, and .74 reliability. These results exhibit moderate reliability. 

Because of the subjectivity possible in grading the writing and speaking portions of the 
English test, intrarater reliability had to be calculated. Each of the four instructors graded the writing 
and speaking portions of the English tests twice, the second grading occurring about one week after 
the first grading (see appendices G and H for speaking and writing scoring guides). The speaking 
interviews were recorded and played back only once during the second grading. The first grading 
and second grading were compared using the Cronbach alpha (see tables 4 and 5). 
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Table 4: Descriptive Statistics and Intrarater Reliability for 
First Scoring of English Test Writing and Speaking Sections 


Statistics 


Writing Pretest 


Writing Posttest 


Speaking Pretest 


Speaking Posttest 


N 


293.00 


293.00 


293.00 


293.00 


k 


3.00 


3.00 


5.00 


5.00 


M 


.95 


2.66 


1.01 


3.95 


Low 


.00 


1.00 


0.00 


0.00 


High 


3.00 


3,00 


5.00 


5.00 


SD 


1.11 


.63 


1.50 


1.44 


Cronbach Alpha 


.96 


.91 


.92 


.95 



Table 5: Descriptive Statistics and Intrarater Reliability for 
Second Scoring of English Test Writing and Speaking Sections 



Statistics 


Writing Pretest 


Writing Posttest 


Speaking Pretest 


Speaking Posttest 


N 


293.00 


293.00 


293.00 


293.00 


k 


3.00 


3.00 


5.00 


5.00 


M 


.94 


2.61 


1.01 


3.91 


Low 


.00 


1.00 


.00 


.00 


High 


3.00 


3.00 


5.00 


5.00 


SD 


1.08 


.66 


1.50 


1.44 


Cronbach Alpha 


.96 


.91 


.92 


.95 



The reliability for the writing portions of the pretest and posttest was .96 and .91, respectively. The 
speaking pretest and posttest were .92 and .95. The intrarater reliability for the writing and speaking 
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portions of the English test was high. 



Statistical Analysis 

To analyze the data, descriptive statistics were calculated for the self-efficacy 
questionnaires and English tests (see tables 3, 6, and 7). 



Table 6: Overall Descriptive Statistics of Self-Efficacy Pre-questionnaire and Post-questionnaire 



Statistics 


Pre=questionnaire 


Post=questionnaire 


N 


293.00 


293.00 


k 


13.00 


13.00 


M 


2.90 


3.58 


Low 


1.00 


1.00 


High 


7.00 


7.00 


SD 


1.18 


1.14 


Cronbach alpha 


.91 


.89 
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Table 7: Descriptive Statistics for Self-Efficacy Pre-Questionnaire and Post-Questionnaire Items'*' 



Questions 


N 


Low 


High 


Mean 


Std. Deviation 


Qi 


312 


1.00 


7.00 


2.89 


1.08 


*Q1 


312 


1.00 


7.00 


3.27 


1.02 


Q2 


312 


1.00 


7.00 


3.26 


1.14 


*Q2 


312 


1.00 


7.00 


4.01 


1.06 


Q3 


312 


1.00 


7.00 


3.00 


1.03 


*Q3 


312 


1.00 


7.00 


3.59 


1.00 


Q4 


312 


1.00 


7.00 


2.60 


1.28 


*Q4 


312 


1.00 


7.00 


3.17 


1.08 


Q5 


312 


1.00 


7.00 


2.82 


1.09 


*Q5 


312 


1.00 


7.00 


3.56 


1.07 


Q6 


312 


1.00 


7.00 


2.85 


1.22 


*Q6 


312 


1.00 


7.00 


3.67 


1.18 


Q7 


312 


1.00 


7.00 


2.21 


1.17 


*Q7 


312 


1.00 


7.00 


3.10 


1.21 


Q8 


312 


1.00 


7.00 


2.69 


1.23 


*Q8 


312 


1.00 


7.00 


3.58 


1.32 


Q9 


312 


1.00 


7.00 


3.30 


1.22 


*Q9 


312 


1.00 


7.00 


4.05 


1.16 


Q10 


312 


1.00 


7.00 


3.21 


1.15 


*Q10 


312 


1.00 


7.00 


3.56 


1.11 


Qll 


312 


1.00 


7.00 


3.00 


1.28 


*Q11 


312 


1.00 


7.00 


3.72 


1.35 


Q12 


312 


1.00 


7.00 


2.96 


1.21 


*Q12 


312 


1.00 


7.00 


3.79 


1.21 


Ql3 


312 


1.00 


7.00 


3.31 


1.27 


*Q13 


312 


1.00 


7.00 


3.54 


1.16 



’’’Pre- Questionnaire Questions have no asterisk, and post-questionnaire questions have an asterisk. 



Differences were measured by a paired t-test, with an alpha level of .05. 
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Results 



The differences between the pre-questionnaire and post-questionnaire scores were 
significant at p < .05 (see tables 8 and 9). 



Table 8: Matched T-Test of Pre-Questionnaire and Post-Questionnaire 


Statistic 


Pre-Questionnaire and Post-Questionnaire 


Mean 


-.60 


Standard Deviation 


1.09 


Standard Error of Mean 


6.60 


Lower Confidence Interval of Difference ( .05) 


-.73 


Upper Confidence Interval of Difference (.05) 


-.47 


df 


276.00 


t 


-.91* 


*p<05 



Table 9: Matched T-Test of Pretest and Posttest 


Statistic 


Pretest and Posttest 


Mean 


-10.23 


Standard Deviation 


6.54 


Standard Error of Mean 


.38 


Lower Confidence Interval of Difference ( .05) 


-10.98 


Upper Confidence Interval of Difference (.05) 


-9.48 


df 


291.00 


t 


-26.73* 


*p<05 



Also, the differences between the English pretest and posttest scores were significant at p < .05. 
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Discussion 



The self-efficacy questionnaire and English test were reliable and valid. Also, students’ 



self-efficacy and English ability increased over the period of instruction (see figures 1 and 2 for 



graphs and see appendices E and F for examples of increased writing and speaking ability). 



Figure 1: Comparison of Proportion Means: Pretest (1) and Posttest (2) 
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The results are in agreement with self-efficacy research that shows that students’ self-efficacy and 
achievement can be raised through teaching. Future research should examine the correlation and 
casual relationship between self-efficacy and L2 achievement. 
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Appendix A: English I Objectives 
English I Course Objectives 

Students will demonstrate that they understand specific details of a 
conversation between two people meeting for the first time 

Students will ask questions when meeting someone for the first time 
Students will fill out customs forms 

Students will demonstrate that they understand the meaning of words on signs in 
a plane 

Students will demonstrate that they know that is, are, and am are often reduced 
in informal conversation 

Students will demonstrate that they recognize whether pitch goes up or down at 
the end of a sentence 

Students will use hesitation phrases when telling personal information 
Students will demonstrate that they recognize the difference in frequency 
of English use in 1 2 countries 

Students will change negative ideas into positive ideas and plan positive 
actions 

Students will demonstrate that they recognize things in common between two 
people in a conversation 

Students will ask yes-no questions about likes/dislikes 
Students will write an order for food and drinks (at least 35 words), and 
demonstrate their reading comprehension of an order for food and drinks 
Students will demonstrate that they understand the meaning of words on signs at 
the beach 

Students will recognize politeness levels in commands 
Students will pronounce and distinguish /I/ and /r/ sounds 
Students will put instructions in order and insert action order phrases 
Students will demonstrate that they recognize the meaning of five gestures 
Students will write in Japanese about a successful experience 

Students will demonstrate that they understand directions and will find places on 
a map 
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Students will give directions using a map 

Students will write directions from school to their residence (at least 45 
words), and will demonstrate their reading comprehension of directions from 
school to someone else’s residence 

Students will demonstrate that they understand the vocabulary used in directions 
Students will use if to describe a condition and mention results or actions 
Students will pronounce and demonstrate they understand contrastive stress on 
words in sentences 

Students will use phrases to interrupt when asking for directions 
Students will use proper distance and body language when asking for 
directions 

Students will demonstrate a technique for relaxing 

Students will demonstrate that they recognize who is being described by 
listening to self-introductions 

Students will describe the location of furnishings in an apartment 
Students will write an ad (at least 55 words) describing their ideal 
roommate, and will demonstrate reading comprehension of an ad for a 
roommate 

Students will demonstrate that they understand the names for items of furniture 
Students will use prepositions to describe the location of furniture in an 
apartment 

Students will pronounce and distinguish /i/ and /I/ sounds 

Students will express alternate views using anyway, but then again, even so, 

look at it this way 

Students will demonstrate that they understand two people can experience 
the same thing differently 

Students will fill out a 1-week schedule to see how they spend their time 

Students will demonstrate that they understand a conversation about a student’s 
schedule 

Students will invite one another to an activity 

Students will write in a journal at least 5 minutes, 5 days a week 

Students will use simple and progressive tenses in sentences 

Students will demonstrate that they recognize English is stressed-timed 

Students will show interest while listening by using OK, yes, yeah, uh-huh, and, 
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Chapter Eight 
Listening 



really 

Students will demonstrate that they know people value time differently 
Students will evaluate their sleeping and eating habits, and revise their 
daily time-schedule 

Students will demonstrate that they understand descriptions of extroverts and 
introverts 

Students will describe different personalities 

Students will read someone else’s journal, decide whether the person 

is an introvert or extrovert, and write positive comments; they will also write 

about their own personality (at least 75 words) 

Students will demonstrate that they understand words describing kinds of 
personalities 

Students will complete sentences describing a time in the past 

Students will pronounce word stress in numbers, and demonstrate they recognize 

such stress 

Students will communicate about differences using but, and however 
Students will demonstrate that “good” personality traits may be different 
in different countries 

Students will follow their revised schedule for one week 

Students will demonstrate that they understand descriptions of people 

Students will talk about an ideal person to date 

Students will write about going out on an ideal date (at least 75 words), 

and demonstrate their reading comprehension of a description of an ideal date 

Students will demonstrate that they understand the meaning of dating 

expressions 

Students will demonstrate that they recognize the order of events 

Students will pronounce and distinguish s and sh sounds 

Students will give reasons for not doing something using because and the reason 

why 

Students will demonstrate they know how to act in five dating situations 
Students will give praise to, and receive praise from a classmate for 
following parts of the revised schedule 

Students will demonstrate that they understand personal information of 




party-goers 



Speaking 

Reading/ 

Writing 

Vocabulary/Syntax 

Pronunciation 

Discourse 

Society/Culture 
Body/ Mind 



Students will give false personal information at a party 
Students will write personal information from a party (at least 75 words), 
and demonstrate reading comprehension of another’s personal information 
Students will complete sentences using modals 

Students will pronounce and demonstrate they recognize reductions in informal 
speech 

Students will demonstrate that they recognize politeness levels for 

requests/advice using modals 

Students will converse with someone at a party 

Students will set goals for speaking with others 



Note: English I test guides for scoring speaking and writing skills, for the most part, require an 
evaluation of student performance in terms other than explicit course objectives. For instance, the 
guide for scoring writing-in addition to gauging an essay against an explicit course objective for a 
length of 75 words--requires an evaluation of essay organization and topic relevance, neither of 
which are stated explicitly in the objectives. However, we understand organization and relevance 
as implied objectives because students are always assigned certain topics for their essays, and 
students are expected to write so their writing can be understood: the more organized an essay, the 
less confusing an essay is and the easier it is to understand. 



So too, for speaking objectives: spontaneity of response, topic relevance, tune taken up in pauses, 
number of English words, and number of intrusive Japanese words are all criteria against which the 
speaking portion of the test is judged. Not one of these criteria is stated explicidy as a course 
objective. Rather, they are implicit objectives relating to a holistic impression of a student’s 
fluency in English; and in satisfying explicit course objectives for speaking, students are expected to 
have met implied course objectives as well. 
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Appendix B: English I Test 



English I Exam 



Listening (3 points each) 

Listen to the conversation. Choose the best answer. 

1 . Why is the woman going to Florida? 

a. to study b. business c. vacation 

2. Where is the woman from in Japan? 

a. Tokyo b. Osaka c. Okinawa 

3 . Has the woman been to Florida before? 

a. Yes b. No 

4. Does the woman like the ocean? 

a. Yes b. No 

5. Can the woman swim? 

a. Yes b. No 

Pronunciation (2 points each) 

Choose whether the pitch goes up or down at the end. 

6. a. up b. down 

Listen. Choose the correct word you hear. 

7. a. light b. right 

8. a. beat b. bit 

Listen. Choose the stressed word. 

9. Go straight on Manning Street and turn left. 

a. b. c. d. 





Reading (5 points each) 

Read the description of this man’s ideal roommate. Answer the questions below. 

I want a roommate who is male. I want a roommate who lets me smoke in my apartment. I 
want someone neat. He should not be loud late at night. I also want someone who is kind and not 
outgoing. I had an outgoing roommate before, and I almost went crazy because he was always 
talking. I never had any quiet time for myself. I don’t like impulsive people because they seem to get 
into trouble. This is my idea of an ideal roommate. 

1 0. Which roommate should this person choose? 

a. A shy woman who smokes. 

b. A shy man who smokes. 

c. An outgoing man who smokes. 

d. An outgoing man who does not smoke. 

Read what Maria wrote about herself. Answer the question below. 

I have many friends in Sapporo. Sometimes we do crazy things. I like people. In America, I learn 
English at parties but not in the classroom. 

1 1 . Maria is . 

a. shy b. outgoing c. cautious 

Read the directions. Look at the map. Choose where the place is. 

12. Go straight. Turn right on Bird=of=Paradise Road. Then turn left on Damon Lane. You’ll see it on 
the right. 

a. School of Education b. Business School c. Classrooms d. Plant Facilities e. Hale 5 




(Templin & Guile, 2000) 
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Vocabulary (2 points each) 

Match each word with the best picture. 

13. No smoking 

14. Fasten seat belts 





c 





K 



(Templin & Guile, 2000) 



Match the illustration to the word. 
15. 




a. hard-working 



b. lazy 



16. 







(Tettiplin & Guile, 2000) 



a. serious 



b. fun-loving 
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Syntax (2 points each) 

17. Complete the sentence. 

If I get up late, . 

a. I’ll be late for school 

b. because I’ll be late for school 

c. when I’ll be late for school 

d. Both b and c are correct 

18. The VCR is the TV. 




a. beside b. in c. on d. under 

Choose the correct word to complete the sentence. 

1 9. At a time in the past 

Jorve a new hat last summer, 

a. bought b. buy c. buys d. have bought 

20. Read the sentence. Which happened first (a or b)? 

a b 

Before playing a computer game. Roberto started to study . 

Discourse (2 points each) 

2 1 . When you are thinking about what to say, use: 

a. Great b. But then again c. Even so d. Hmm 

22. When you explain the steps of an activity, use: 

a. Excuse me b. First of all c. Hmm d. Anyway 

23. Choose the correct phrase to change the negative statement into a positive statement. 
Sachiko: Living in a dorm is difficult. 

Leanni: , living in the dorm is cheaper. 

a. Uh b. But then again c. Next d. Sorry 
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24. Koji gives a reason why he did not follow parts of his schedule. Which phrase should he begin 
with? 

a. Because b. But c. I could d. Really 

Society & Culture (2 points each) 

25. When you ask a stranger for directions, what body language should you use? 

a. Smile and use gestures. 

b. Smile but don’t use gestures, 

c. Use gestures but don’t smile. 

d. Don’t smile and don’t use gestures. 

26. Which of the following is true? 

a. People always experience the same thing differently. 

b. People never experience the same thing differently. 

c. Sometimes people experience the same thing differently. 

27. Which group of people uses this proverb: Time is money? 

a. North Americans b. Ethiopians c. American Indians d. Saudi Arabians 

28. On a date, at the movie theater, who usually pays? 

a. The man and woman share the expense. 

b. the man 

c. the woman 

Writing 

-Write 75 words (or more) about an ideal date. 

-Write the number of words you used. 



[DO NOT WRITE HERE. WRITE ON THE BACK OF THE ANSWER SHEET,] 
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Appendix D: Self-Efficacy Questionnaire (English Translation) 

English Self-Efficacy Scale (3/1/01) 

This questionnaire is designed to help your instructors get a better understanding about what you 
find difficult about English. Please indicate your opinions about each of the statements below by 
circling the appropriate number. Your answer will be kept confidential and will not affect your grade. 
Please take this seriously, 

1 . How well can you understand specific details of a conversation between two people meeting for 
the first time? 



1 2 3 

Not well at all Not too well 


4 


5 

Pretty well 


6 


7 

Very well 


2. How well can you demonstrate that you recognize whether pitch goes up or down at the end of a 
sentence? 

1 1 1 1 1 1 1 


1 2 3 

Not well at all Not too well 


4 


5 

Pretty well 


6 


7 

Very well 


3. How well can you talk about yourself? 


I 




I 


1 


1 2 3 

Not well at all Not too well 


4 


5 

Pretty well 


6 


7 

Very well 



4. How well can you describe the kind of person you would like to date? 



1 



2 



3 



4 



5 



6 



7 



Not well at all Not too well 



Pretty well 



Very well 
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5. How well can you comprehend a printed ad for a roommate? 



1 2 3 4 5 6 7 

Not well at all Not too well Pretty well Very well 

6. How well can you understand writing that describes whether someone is an extrovert or 
introvert? 



12 3 4 5 6 7 

Not well at all Not too well Pretty well Very well 

7. How well can you write about your ideal person to date using at least 75 words? 



12 3 4 5 6 7 

Not well at all Not too well Pretty well Very well 

8. How well can you write about yourself using at least 75 words? 



1 2 3 4 5 6 7 

Not well at all Not too well Pretty well Very well 

9. How well can you understand signs in an airplane? 



1 


2 


3 


4 


5 


6 


7 


Not well at all 




Not too well 




Pretty well 




Very well 



10. How well can you recognize which event happened first out of two events in the past? 



12 3 4 5 6 7 

Not well at all Not too well Pretty well Very well 
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1 1 . How well can you pronounce and distinguish /i/ (eat) and /!/ (it) sounds? 



12 3 4 5 6 7 

Not well at all Not too well Pretty well Very well 

1 2. How well can you show interest by using OK, yes, yeah, uh-huh, and really ? 



1 2 3 4 5 6 7 

Not well ot nil Not too well Pretty well Very well 



13. How well can you understand that two people can experience the same thing differently? 



Not well at all 



3 

Not too well 



5 

Pretty well 



7 

Very well 
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Appendix E: Pretest and Posttest Writing Results for Student A and Student B 



Pretest 

Student A 

[Wrote nothing] 

Student B 
[Wrote nothing] 

Posttest 

Student A 

First, we go to the “Makku” for eat “Poteto”. Second, we go to the movie theater for watch movie of 
“AT’. We go to the “Gulume Kaiten” for eat dinner of Sushi. After the dinner is drive. My car drive 
Naha with my lady. After the drive go to Chatan twan for play “Kanransha”. We go to “Sega” for 
play game of “Jack Pot”. After the “Sega” go to Nago city for go home. The date is end. 

Student B 

I think that ideal date is very prity girl and small and good smile. I like love and peace so 
smile is very like. I don’t like don’t smile girl, 

And I love can cook girl. Because, I am very hungry. If I don’t eatting, I will be die. I 
don’t wanna die. So I love can cook girl. Please help me. 

When I am going to go to date, I don’t watch movie. Movie is kill time. So I watch movie. 
I want talk with my girl friend. 
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Appendix F: Pretest and Posttest Speaking Results for Student A and Student B 



Pretest 

Student A 
[Said nothing] 

Student B 
[Said nothing] 

Posttest 

Student A 

Uh-huh. Let me see. Like. I like baseball. Dislike. Mm. Uh, let me see. OK, OK. Driver drive not 

like. 

Student B 

My name is [deleted]. I am 18 years old. I’m Meio University student. My major is tourism. And 
I’m join softball club. I’m not good at... I’m not a good player. But I like softball very much. 
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Appendix G: Writing Score Guide 



English I Writing Score Guide for Exam A & B (5 points each) 



W29 


Essay Length 


a 


75 words or more. 


b 


50-74 words. 


c 


25-49 words. 


d 


0-24 words, 


W30 


Organization 


a 


Weii-ordered. 


b 


Incomplete order. 


c 


No order. 


d 


Too unclear to understand. 


W31 


Topic Relevance 


a 


75-100% related to topic. 


b 


50-74% related to topic. 


c 


25-49% related to topic. 


d 


0-24% related to topic. 



Note: Score of a (a or b for item 30) is passing for that item. 
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Appendix H: Speaking Score Guide 

English I Speaking Exam (A) Question 

Tell me about your likes and dislikes. 



English I Speaking Exam (B) Question 
What kind of person would you like to date? 



English I Speaking Score Guide for Exam A & B (3 points each) 



S32 


Clarification 


a 


Answers the auestion without needing clarification. 


b 


Asks to repeat the question 1-2 times. 


C 


Asks to repeat the question 3 or more times. 


d 


Passes the question: I don’t know, no, etc. 


e 


Does not speak English or is too unclear to be understood. 


S33 


Using Non-English Words 


a 


Speaks only English. 


b 


Speaks 1-2 non-English words. 


c 


Speaks 3-4 non-English words. 


d 


Speaks 5 or more non-English words. 


e 


Does not speak English. 


S34 


Total Length of Pauses (silence without hesitation phrases, etc.) 


a 


Pauses 0-1 second. 


b 


Pauses 2-3 seconds. 


c 


Pauses 4-5 seconds. 


d 


Pauses 6-7 seconds. 


e 


Pauses 8 or more seconds. 


S35 


Length of Utterance 


a 


15-or-more-word response. 


b 


10-14 word response. 


c 


5-9 word response. 


d 


1 -4 word response. 


e 


Does not speak English. 


S36 


Topic Relevance 


a 


All related to topic. 


b 


More than 1/2 related to topic. 


C 


Less than 1/2 related to topic. 


d 


Does not speak English 



Note: Score of a or b (bolded) is passing for that item. 
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